MLE-24717 Some Polaris fixes #565
Conversation
rjrudin
requested review from
BillFarber,
anu3990 and
stevebio
as code owners
|
Copyright Validation Results ✅ Valid Files
✅ All files have valid copyright headers! |
There was a problem hiding this comment.
Pull Request Overview
This PR applies several fixes to improve the Polaris functionality in the MarkLogic Spark connector. The changes focus on defensive programming practices and edge case handling.
- Added divide-by-zero protection for file partition creation
- Enhanced JSON deserialization with null safety checks
- Added test coverage for zero partition edge cases
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| FileUtil.java | Added validation to prevent divide-by-zero when numPartitions is <= 0 |
| JsonRowDeserializer.java | Added null safety check for JSON parsing results |
| MakeFilePartitionsTest.java | Added test case to verify zero partition handling |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| public InternalRow deserializeJson(String json) { | ||
| return this.jacksonParser.parse(json, this.jsonParserCreator, this.utf8StringCreator).head(); | ||
| var parsedResult = this.jacksonParser.parse(json, this.jsonParserCreator, this.utf8StringCreator); | ||
| Objects.requireNonNull(parsedResult, "The output from parsing the JSON should never be null"); |
There was a problem hiding this comment.
The error message should be more specific about what happens when null is encountered and provide actionable information for debugging, such as 'Failed to parse JSON input - received null result from JacksonParser'.
| Objects.requireNonNull(parsedResult, "The output from parsing the JSON should never be null"); | |
| Objects.requireNonNull(parsedResult, | |
| "Failed to parse JSON input - received null result from JacksonParser. " + | |
| "Input (truncated to 100 chars): " + | |
| (json == null ? "null" : json.substring(0, Math.min(json.length(), 100))) | |
| ); |
|
No description provided.